{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "keras.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [
        "Tce3stUlHN0L"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.0"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "tuOe1ymfHZPu",
        "colab": {}
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MfBg1C5NB3X0"
      },
      "source": [
        "# TensorFlow ile egitimin dagitimi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "r6P32iYYV27b"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/tr/r1/tutorials/distribute/keras.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/tr/r1/tutorials/distribute/keras.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SNsdT6sJihFa"
      },
      "source": [
        "Note: Bu dökümanlar TensorFlow gönüllü kullanıcıları tarafından çevirilmiştir.\n",
        "Topluluk tarafından sağlananan çeviriler gönüllülerin ellerinden geldiğince\n",
        "güncellendiği için [Resmi İngilizce dökümanlar](https://www.tensorflow.org/?hl=en)\n",
        "ile bire bir aynı olmasını garantileyemeyiz. Eğer bu tercümeleri iyileştirmek\n",
        "için önerileriniz var ise lütfen [tensorflow/docs](https://github.com/tensorflow/docs)\n",
        "havuzuna pull request gönderin. Gönüllü olarak çevirilere katkıda bulunmak için\n",
        "[docs-tr@tensorflow.org](https://groups.google.com/a/tensorflow.org/forum/#!forum/docs-tr)\n",
        "listesi ile iletişime geçebilirsiniz."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xHxb-dlhMIzW"
      },
      "source": [
        "## Genel Bakis\n",
        "`tf.distribute.Strategy` API'si TF programinizi birden cok bilgisayar/GPU'ya dagitmak icin bir taslak saglar. Buradaki amac kullanicilarin var olan modellerini ve kodlarini olabildigince ayni tutarak dagitim yapmalarini saglamaktir.\n",
        "\n",
        "Bu rehber `tf.distribute.MirroredStrategy` taktigini kullanarak ayni makinedeki birden fazla GPU'da ayni \"graph\"i kullanarak eszamanli egitim yapmamizi saglar. Isin ozunde, bu taktik modelinizin degiskenlerini butun islemcilere kopyalar. Daha sonra, [all-reduce](http://mpitutorial.com/tutorials/mpi-reduce-and-allreduce/) yontemi ile butun islemcilerden gelen degisimleri birlestirir ve elde edilen bu degeri varolan butun kopyalara uygular.\n",
        "\n",
        "`MirroredStrategy` taktigi TensorFlow'un icindeki dagitim taktiklerinden sadece biridir. Diger taktikler hakkinda bilgi edinmek icin bu rehbere goz atabilirsiniz: [dagitim taktikleri rehberi](../../guide/distribute_strategy.ipynb).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MUXex9ctTuDB"
      },
      "source": [
        "### Keras API\n",
        "Bu ornekteki modeller ve egitim donguleri `tf.keras` API'si kullanilarak olusturulmustur. Eger, kendinize ozgu bir egitim dongusu olusturmak istiyorsaniz, [buraya](training_loops.ipynb) goz atiniz."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Dney9v7BsJij"
      },
      "source": [
        "## Gereksinimleri indirelim"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "r8S3ublR7Ay8",
        "colab": {}
      },
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "74rHkS_DB3X2",
        "colab": {}
      },
      "source": [
        "# TensorFlow kutuphanesini getirelim\n",
        "import tensorflow as tf  #gpu\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "import os"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "hXhefksNKk2I"
      },
      "source": [
        "## Veri setini indirelim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "OtnnUwvmB3X5"
      },
      "source": [
        "MNIST veri setini [TensorFlow Datasets](https://www.tensorflow.org/datasets) kullanarak indirip yukleyelim."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lHAPqG8MtS8M"
      },
      "source": [
        "`with_info` degiskenini `True` olarak belirtmek metadata'nin butun veri setini kapsadigini belirtir. Bu `metadata` `ds_info` degiskeninde saklanmaktadir. \n",
        "Bu metadata nesnesi egitim ve test orneklerinin sayisini da icinde barindirir.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iXMJ3G9NB3X6",
        "colab": {}
      },
      "source": [
        "datasets, ds_info = tfds.load(name='mnist', with_info=True, as_supervised=True)\n",
        "mnist_train, mnist_test = datasets['train'], datasets['test']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GrjVhv-eKuHD"
      },
      "source": [
        "## Dagitim taktigimizi tanimlayalim\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "TlH8vx6BB3X9"
      },
      "source": [
        "Bir `MirroredStrategy` nesnesi olusturalim. Bu sayede dagitimi kontrol edebilir ve modelimizi icinde olusturabilecegimiz bir ortam yonetmeni (`tf.distribute.MirroredStrategy.scope`) saglamis oluruz. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "4j0tdf4YB3X9",
        "colab": {}
      },
      "source": [
        "strategy = tf.distribute.MirroredStrategy()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "cY3KA_h2iVfN",
        "colab": {}
      },
      "source": [
        "print ('Number of devices: {}'.format(strategy.num_replicas_in_sync))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lNbPv0yAleW8"
      },
      "source": [
        "## Input pipeline kuralim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "psozqcuptXhK"
      },
      "source": [
        "Eger bir model birden fazla GPU ile egitiliyorsa, grup boyutleri ayni oranda arttirilmalidir ki islemci gucunu verimli sekilde kullanabilelim. Ayni zamanda ogrenme hizinin da GPU miktarina gore ayarlanmasi gerekir. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "p1xWxKcnhar9",
        "colab": {}
      },
      "source": [
        "# 'ds_info.splits.total_num_examples' metodunu kullanarak da orneklerin sayisini bulabilirsiniz.\n",
        "\n",
        "num_train_examples = ds_info.splits['train'].num_examples\n",
        "num_test_examples = ds_info.splits['test'].num_examples\n",
        "\n",
        "BUFFER_SIZE = 10000\n",
        "\n",
        "BATCH_SIZE_PER_REPLICA = 64\n",
        "BATCH_SIZE = BATCH_SIZE_PER_REPLICA * strategy.num_replicas_in_sync"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0Wm5rsL2KoDF"
      },
      "source": [
        "Piksel degerleri, ki bunlar aslinda 0-255 arasindadir, [normallestirilerek 0 ile 1 arasinda bir degere indirgenmelidir](https://en.wikipedia.org/wiki/Feature_scaling). Bu ingirdenme olcegini bir fonksiyon ile tanimlayalim."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Eo9a46ZeJCkm",
        "colab": {}
      },
      "source": [
        "def scale(image, label):\n",
        "  image = tf.cast(image, tf.float32)\n",
        "  image /= 255\n",
        "\n",
        "  return image, label"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WZCa5RLc5A91"
      },
      "source": [
        "Simdi bu fonksiyonu egitim ve test verisine uygulayalim. Sonrasinda egitim verisini karistiralim ve [egitim icin gruplayalim](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#batch).\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "gRZu2maChwdT",
        "colab": {}
      },
      "source": [
        "train_dataset = mnist_train.map(scale).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)\n",
        "eval_dataset = mnist_test.map(scale).batch(BATCH_SIZE)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "4xsComp8Kz5H"
      },
      "source": [
        "## Modeli olusturalim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "1BnQYQTpB3YA"
      },
      "source": [
        "Keras modelimizi `strategy.scope` ortaminda olusturalim ve toparlayalim."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "IexhL_vIB3YA",
        "colab": {}
      },
      "source": [
        "with strategy.scope():\n",
        "  model = tf.keras.Sequential([\n",
        "      tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),\n",
        "      tf.keras.layers.MaxPooling2D(),\n",
        "      tf.keras.layers.Flatten(),\n",
        "      tf.keras.layers.Dense(64, activation='relu'),\n",
        "      tf.keras.layers.Dense(10, activation='softmax')\n",
        "  ])\n",
        "\n",
        "  model.compile(loss='sparse_categorical_crossentropy',\n",
        "                optimizer=tf.keras.optimizers.Adam(),\n",
        "                metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "8i6OU5W9Vy2u"
      },
      "source": [
        "## Bildirim fonksiyonlarini tanimlayalim\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YOXO5nvvK3US"
      },
      "source": [
        "Burada kullanilan bildirim fonksiyonlari sunlardir:\n",
        "*   *Tensorboard*: Bu geri arama Tensorboard'lar icin kayit tutar. Bu kayitlar ile bir grafik olusturabiliriz.\n",
        "*   *Model Checkpoint*: Bu geri arama her devir sonunda modeli kaydeder.\n",
        "*   *Learning Rate Scheduler*: Bu geri aramayi kullanarak, ogrenme hizinin her devir ya da gruptan sonra degismesini programlayabilirsiniz.\n",
        "\n",
        "Daha aciklayici olmasi icin, 'ogrenme hizini' bir geri arama ile kitapciga yazdirabilirsiniz."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "A9bwLCcXzSgy",
        "colab": {}
      },
      "source": [
        "# Kontrol noktalarini kaydetmek icin bir kontrol noktasi dosyasi olusturalim.\n",
        "\n",
        "checkpoint_dir = './training_checkpoints'\n",
        "# Kontrol noktasi dosyalarinin ismi\n",
        "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt_{epoch}\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "wpU-BEdzJDbK",
        "colab": {}
      },
      "source": [
        "# Azalan ogrenme hizi icin bir fonksiyon\n",
        "# Ihtiyaciniz olan butun azalma fonksiyonlarini kullanabilirsiniz.\n",
        "def decay(epoch):\n",
        "  if epoch < 3:\n",
        "    return 1e-3\n",
        "  elif epoch >= 3 and epoch < 7:\n",
        "    return 1e-4\n",
        "  else:\n",
        "    return 1e-5"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "jKhiMgXtKq2w",
        "colab": {}
      },
      "source": [
        "# Her devrin sonunda LR'i (ogrenme hizini) yazdirmak icin bir geri arama fonksiyonu.\n",
        "class PrintLR(tf.keras.callbacks.Callback):\n",
        "  def on_epoch_end(self, epoch, logs=None):\n",
        "    print ('\\nLearning rate for epoch {} is {}'.format(\n",
        "        epoch + 1, tf.keras.backend.get_value(model.optimizer.lr)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "YVqAbR6YyNQh",
        "colab": {}
      },
      "source": [
        "callbacks = [\n",
        "    tf.keras.callbacks.TensorBoard(log_dir='./logs'),\n",
        "    tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_prefix,\n",
        "                                       save_weights_only=True),\n",
        "    tf.keras.callbacks.LearningRateScheduler(decay),\n",
        "    PrintLR()\n",
        "]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "70HXgDQmK46q"
      },
      "source": [
        "## Egitim ve yorumlama"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6EophnOAB3YD"
      },
      "source": [
        "Simdi, modelimizi her zamanki gibi egitelim: `fit` yontemini model uzerinde cagirip bu rehberin basinda olusturdugumuz veri setini modele verelim. Bu asama dagitim yapsaniz da yapmasaniz da hep ayni kalacaktir.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "7MVw_6CqB3YD",
        "colab": {}
      },
      "source": [
        "model.fit(train_dataset, epochs=10, callbacks=callbacks)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NUcWAUUupIvG"
      },
      "source": [
        "Asagida gordugunuz gibi, kontrol noktalari hafizaya kaydedilmektedir."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JQ4zeSTxKEhB",
        "colab": {}
      },
      "source": [
        "# Kontrol noktalari dosyasina bakalim.\n",
        "!ls {checkpoint_dir}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "qor53h7FpMke"
      },
      "source": [
        "Modelin nasil calistigini gormek icin, en son kaydedilen kontrol noktasini yukleyip 'evaluate' yontemini test verisinde cagirabilirsiniz.\n",
        "\n",
        "'evaluate' yontemini daha once yaptiginiz gibi uygun veri setlerinde kullanmalisiniz."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JtEwxiTgpQoP",
        "colab": {}
      },
      "source": [
        "model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))\n",
        "\n",
        "eval_loss, eval_acc = model.evaluate(eval_dataset)\n",
        "print('Eval loss: {}, Eval Accuracy: {}'.format(eval_loss, eval_acc))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IIeF2RWfYu4N"
      },
      "source": [
        "Programin sonuclarini, TensorBoard kayitlarini indirerek terminalde gorebilirsiniz. \n",
        "\n",
        "```\n",
        "$ tensorboard --logdir=path/to/log-directory\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "LnyscOkvKKBR",
        "colab": {}
      },
      "source": [
        "!ls -sh ./logs"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "kBLlogrDvMgg"
      },
      "source": [
        "## Kaydedilen modelin _'SavedModel'_ cikartilmasi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Xa87y_A0vRma"
      },
      "source": [
        "Eger \"graph\"larin ve degiskenlerin program disinda kullanilmasini istiyorsaniz, `SavedModel` sizin icin ideal yontem. Bu yontem herhangi bir kapsama bagli olmadan yuklenebir ve herhangi bir platforma da bagli degildir."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "h8Q4MKOLwG7K",
        "colab": {}
      },
      "source": [
        "path = 'saved_model/'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "4HvcDmVsvQoa",
        "colab": {}
      },
      "source": [
        "tf.keras.experimental.export_saved_model(model, path)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vKJT4w5JwVPI"
      },
      "source": [
        "Modeli 'strategy.scope' olmadan yukleyelim."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "T_gT0RbRvQ3o",
        "colab": {}
      },
      "source": [
        "unreplicated_model = tf.keras.experimental.load_from_saved_model(path)\n",
        "\n",
        "unreplicated_model.compile(\n",
        "    loss='sparse_categorical_crossentropy',\n",
        "    optimizer=tf.keras.optimizers.Adam(),\n",
        "    metrics=['accuracy'])\n",
        "\n",
        "eval_loss, eval_acc = unreplicated_model.evaluate(eval_dataset)\n",
        "print('Eval loss: {}, Eval Accuracy: {}'.format(eval_loss, eval_acc))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "8uNqWRdDMl5S"
      },
      "source": [
        "## Sirada ne var?\n",
        "\n",
        "Dagitim taktikleri rehberini [distribution strategy guide](../../guide/distribute_strategy_tf1.ipynb) okuyunuz.\n",
        "\n",
        "Note: `tf.distribute.Strategy` surekli gelistirilmektedir ve yakin zamanda yeni ornekler ve rehberler buraya eklenecektir. Lutfen bu yontemleri deneyiniz. Sizden gelen yorumlara her zaman acigiz. Bu yorumlari buraya yazabilirsiniz [issues on GitHub](https://github.com/tensorflow/tensorflow/issues/new)."
      ]
    }
  ]
}